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Courses^ instruction, arrangements, and student 
outcomes of the Monterrey Tech Staff Development Program were 
evaluated. The program was designed to assist faculty and 
administrators of the Monterrey Institute of Technology strengthen 
their qualifications through a master degree program in education 
offered in Mexico City by regular members of the Oniversity of 
Alabama faculty^ Each course and its instructor were assessed using 
the NCS Student Survey of Course/Instructor, which is appended, 
additional questions regarding students* perceptions of their own 
effozts and the physical arrangements made for the course were added 
to the instrument* Student outcomes were examined in terms of 
achlevenentr written ejtpression in English, and attitudes about their 
jobs« Pre- and post-testing cf student achievement were undertaken 
using items submitted by the course instructors, while written 
expression in English was assessed using pre- and post-course writing 
sampli^s* a semantic differential instrument, which is appended, was 
rusea t^ evaluate student job attitudes. Additionally, Alabama 
professors were asked to complete a short questionnaire, which is 
•appended> concerning travel arrangeirents, housing, local 
transportation^ and course arrangement s» Besults indicate that 
Instructors and administrators participating in the master" s degree 
program significantly improved in the areas of achievement, written 
expression in English, and attitude. The courses and instructors were 
rated good to very good by the students. The arrangements for travel 
and the courses seemed to be more than adequate. (S«) 
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Evaluation of the Monterrey Tech Staff 
Development Program in Mexico City 
1979-80 

The purpose of the Monterrey Tech Staff Deve?,opinent Program was to assist 
the Monterrey Tech faculty and cjaministrators strengthen tlieir educational 
qualifications. This was accomplished by offering selected Monterrey Tech 
faculty and administrators a master's degree program in ed.ucation in Mexico 
City by regialar members of The University of Alabama ^:aculty. 

Personnel at Monterrey Tech could choose betweer two emphases in their 
master's degree programs* These were administration or instruction. Table 1 
provides a list of the courses offered , their instructors r and the dates. 
All courses were taught in English and students were expected to complete a 
comprehensive examination on tlie subject matter ^iven in English. 

Twenty-five students were scheduled to start the program. Records 
indicate that 23 registered for the first course, and 21 students earned 
their master's degrees. 

The remainder of this r:sport provides a comprehensive evaluation of this 
effort. Evaluation of the courses, the instruction, the arrangements, and 
student outcomes are addressed in the report. 

Method 

The objective of >'9 evaluation was to assess the courses and instruction 
as well as student outcomes. Further, the logistics of offering an entire 
master's degree program in Mexico City is addressed. 
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Table 1 

Course Offerings in Monterrey Tech Program 



Core Courses 
(Taken by both Instructors 

and Administrators) Instructor Dates 





590 


Research Methods 


Peseau 


June, 


1979 


AHE 


660 


Curriculum Development 


Peseau 


June , 


1979 


AHE 


640 


Admin* of Higher Education 


Duncan 


July , 


1979 


AHE 


620 


Student in Higher Education 


Duncan 


July, 


1979 


Courses for Administrators Only 








AAP 


650 


Educational Planning I 


Essex 


Oct. , 


1979 


AB3E 


63a 


Law in Higher Education 


Mexican Natl. 


March , 


1980 


AHE 


641 


Personnel in Higher Education 


Dunccin 


June/. 


1980 


AHE 


645 


Institutional Analysis 


Sutton 


July, 


1980 


AHE 


655 


Finance of Higher Education 


Dxincan 


June, 


1930 


AHE 


691 


Clinical Experience 


Sutton 


July, 


1980 




Coutses for Instructors Only 








BEP 


563 


Contr. of Psych, to Teaching 


Joslin 


April, 


. 1980 


AHE 


599 


Clinical Internship 


Anderson 


July, 


1980 


AHE 


665 


College Instruccion 


Vandermeer 


Junei 


1980 


AHE 


670 


Measurement and Evaluation 


Vandermeer 


June, 


1980 


ASC 


603 


Adv. Study in Curriculum Impr. 


Rockarts 


Nov, , 


1979 


ASC 


572 


Classroom Observation 


Anderson 


July, 


1980 



Course Evaluation 

Each course and its instructor was assessed using the NCS Student Survey 
of Coxarse/Instructor (see Attachment A) . The instrument has 27 questions which 
the students answer about the course and instructor. Three additional questions 
were -added (see Attachment B) . These questions were designed to solicit the 
students* perceptions of their own efforts and the physical arrangements made 
for the coixrse. 
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Student Outcomes 

Three types of student outcomes were examined. These were student 
achievement, written expression in English, and attitudes about the:'.r jobs. 

Student achievement . The assessment of student achievement was based upon 
student performance on a pretest and posttest developed for that purpose. The 
tests were predominantly of the multiple-choice variety, but did include some 
matching type exercises. Separate tests were given to administrators and 
instructors. 

I'he tests were developed from items submitted by the course instructors. 
Each instructor was asked to submit 20 multiple-choice items for each course 
they would teach based upon their course objectives. The evaluators reviewed 
these items for technical adequacy and chose 10 from each course for the tests. 
The instructor** s test had 90 items while the administrator's test had 80. 
There were no questions from the clinical courses or from the law course 
taught by a Mexican national. The tests were then reviewed by the instructors 
in the program for content validity. The small niomber of participants in the 
program made an assessment of reliability difficult; however , conservative 
estimates of the reliabilities using approximation procedures (Hills, 1976) 
were .63 and .67 respectively for the instructor's and administrator's test, 

written expression in English . This aspect of student achievement was 
evaluated through the assessment of pre and post writing samples collected 
from the students in the program. A set of instructions for collecting these 
writing samples were provided for the professors (example of instructions for 
collecting posttest writing samples can be found in Attachment C) . 

The pre and post writing samples were paired, labeled "A" and "B" 
irrespective of which v;as pre or post, and evaluated on four bases to determine 
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which was better in each set according to general Standard Written English 
characteristics. The foxzr bases of evaluation were: 

1, Holistic Reading . According to formulae developed by 
such researchers as Paul Diederich, Lee Odell, and 
Charles Cooper, this is an evaluation based upon 
isnpression of organization, coherence, effective 
support/development of thesis or response to question, 
and general readability. 

2, Manuscript error count per lOO'-word passage . Based upon 
the standard conventions of printed English given in a 
variety of handbooks and style manuals, this evaluation 
was to detect relative improvement in correct use of such 
conventions as idiomatic usage, punctuation, spelling, 
subject-verb and pronoun-referent agreement, avoiding 
fragment sentences / garbles, run-ons, and the like, A 
particular eye was kept for second-language interference 
in the form of lexical and syntactical elements. 

3 , C ount for number of words per T-unit per 100-word passage . 
This evaluation was a check for growth or improvement in 
syntactic maturity (ability to generate more complex 
sentence structures) using the T-unit measure developed 
by Kellogg Hunt. The T-unit, which consists of any 
independent clause with all of its modifying words or 
structures, is an efficient reflector of sentence complexity. 

4- Composite impression based upon summary of results from 
1-3 above . In all but one case, results from evaluations 
1-3 aibove were not uniform, necessitating composite 
evaluation to determine which essay was the better. 
(Voss, 1980, p. 1) 

The pre to post changes were compared using standard statistical procedures 
(i.e., paired t-tests and chi sopiare analysis). 

Student attitudes about their jobs . The attitudes students held about 
their jobs were assessed on a pre and post basis using a semantic differential 
Csee Attachment D for instrument) . The 12 bipolar adjectives in the semantic 
differential were chosen from among those representing the "evaluative scale" 
in Osgood's research (Osgood, Suci, & Tannenbaum, 1957). The stimulus was 
"My job" so that the students* attitudes about their jobs were being assessed. 
The pre to post change was analyzed using a paired t-test. 
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Additional Evaluation 

Additional data were collected to assess logistical problems which might 
accompany the delivery of instruction approximately 1400 miles (2250 kilometers) 
from the campus- Each of the Alabama professors was asked to complete a short 
questionnaire (Attachment E) . This questionnaire solicited their opinions 
regarding travel arrangements, housing, local transportation, and course 
arrangements. It also provided space for the course instructors to add any 
comments which they thought appropriate* 

Results 

The results are presented for the evaluation of the courses and instructors, 
student outcomes, and the logistics of teaching so far from campus. Both the 
data and an interpretation cu:e presented. 
course Evaluation 

The mean ratings of instruction on the NCS Student Survey of Course/ 
Instructor (Attachment A) ranged from 3.84 to 4.68 with a median rating of 
4.27. The mean ratings are based on all of the 27 items which were scored on 
the basis of the following scale: 1 represents very poor, 2 represents poor, 
3 represents satisfactory, 4 represents good, and 5 represents very good. Thus, 
all of the mean ratings were in the "good" to "very good" range • 

Upon a closer inspection of the item responses, ratings of the instructors 
seemed to be related very highly to the ratings of the courses • Further, only 
the mean ratings of Items 19 (Your rating of assigned textbooks) and 21 (Your 
rating of supplementary readings) were the lowest and tended to be in the 
"satisfactory" category. This may be attributable to the fact that these 
texts and readings were in English to the displeasure of the students* 



An analysis of the supplementary questions (Attachment B) indicated that 
for the most part, the students believed that they expended efforts ranging 
from very little in some courses to very much in others. Further, students 
felt that: the course schedules were moderately convenient. The classroom 
and equipment were felt to be adequate for the most part. 
Student Outcomes 

The student outcomes are addressed with respect to the achievement levels 
of students, their improvements in written expression in English, and their 
attitudes about their jobs. 

Student achievement > As noted previously, students in both the instructor's 
option and the administrator's option wrote comprehensive objective examinations 
at the beginning and end of the one and one-half year program. The results are 
noted in Table 2. 

Table 2 

Comparison of Pre and Post Achievement 

Pretest Posttest 
Group P.,.^-"' Mean SD Mean SD Gain t 

Instructor 9 43.4 6.97 69.8 8.80 26.4 7.23* 

Administrator 12 41.6 10.08 59.5 4.40 17.9 5.51* 

♦Posttest significantly better than pretest beyond the .001 level. 
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As can be seen in Table 2, gains iii achievement were made by both the 
instructor.^ and administrators in terms of both statistical and practical 
significance. Gains were statistically significant beyond the .001 level for 
both groups. That is^, the probability that real gains were made is at least 
99.9% for both groups* 

Practical significance can be seen in the fact that instructors' and 
administrators' average improvements were 26.3 and 17,9 points respectively. 
These represent improvements of 3.8 and 1.3 standard deviation for instructors 
and administrators respectively. Based on any rule of thumb for assessing 
practical significance, these changes are dramatic. 

Written expression in English . Pre and post samples of written English 
were compared on the basis of holistic reading, error count/ T-unit count, and 
composite impression. A comparison based upon the holistic scoring, error 
count, and T-unit count is presented in Table 3. 

There was a significant reduction in errors from pre to post for both the 
instructors and administrators. Pre to post comparisons of the holistic 
impressions ai^d T-unit counts did not indicate significant changes. 

When the pre writing samples were compared with the the post writing 
samples on the basis of overall impressions , the post samples were rated better 
for 80% of the stu:lcnts. Thus,^ a great majority of the students improved their 
expression in written English, 
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Table 3 

Comparison of Pre and Post Written Expression 



Instructor 



Basis 



n 



Pre 



Mean 



SD 



Post 



Mean 



SD 



Holistic 
Error Coux^t 
T-unit Count 



6 
6 
6 



2,2 
8,7 
212,5 



,34 
2 , 6S 
39,45 



2.1 
6.2 
218.3 



.37 

3,47 
78.80 



-,70 
-4,04* 
,16 



Administrator 



Basis 



Pre 



Mean 



SD 



Post 



Mean 



SD 



Holistic 
Error Count 
T'-'Unit Count 



12 
12 
12 



2.2 -49 
9.8 5.44^ 
218.4 58.83* 



*Change significant at ,05 level. 



2.1 .44 

7.2 4.63 
222.2 85.63 



-1,34 
-2.23* 
,13 



Several complications must be taken into account when interpreting these 
results. They are: 

1. Each pre- and post- essay was actually comprised of two 
separate essays, making it necessary to rate each response 
separately in the holistic reading. Thus, essay "A" called 
for two separate holistic ratings (since two separate topics 
were being responded to), and essay "B" called for two 
separate holistic ratings. Holistic rating is generally 
more reliable when only one topic is treated. 

2. Several students chose to respond in a enumerative , 
"outline" form (in some cases, this form was used on both 
pre- and post- essays). Although such form is efficient 
and probable indicates good orgemizational ability, 
holistic rating is generally intended for evaluating 
continuous prose, and is less reliable in other applications. 

3. Most of the pre- and post- essays were dated, making it 
possible for the evaluator to know which essay was the 
post- essay. In most cases, the post- essay would naturally 
be expected to be the overall better essay. Such knowledge 
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• could have served to prejudice the evaluator. (However, 
there were at least two cases where despite this knowledge, 
the evaluator decided that the pre- essay was the better of 
txie two.) (Voss, 1980, p. 2) 

The general conclusions provided by the English specialist after his 
evaluation was: 

As the data presented previously show, most of the students in 
the Monterrey Tech Program improved their use of Standard Written 
English during their course of study. The preponderance of 
evidence is that the post- essays reflect bette:: organization 
and suppprt, fewer conventional manuscript errors, and greater 
syntactic maturity. The post- essays also show a keener 
perception and understanding of appropriate response to the 
topics. In many cases f the pre- essays contained more personal 
detail and more general response to the "mission" of the program, 
but the post- essays showed greater control of subject, illus- 
tration, and focus. (Vossr 1980, p. 2) 

Student attitudes about jobs . A comparison of the students' attitudes 
about their jobs from the beginning to the end of the program was assessed 
using a semantic differential (Attachment D) . The results are noted in 
Table 4. 



Table 4 



Comparison of Attitudes Pre and Post 









Pre 




Post 










Group 


n 


Mean 


SD 


Mean 


SD 


Gain 


t 


Instructor 


11 


47.5 


4.85 


54. 


9 7 


.82 


7 


.4 


3.65* 


Admini s tra tor 


9 


48.0 


7.26 


50. 


7 8 


.22 


2 


.7 


1.05 



♦Attitude improved at .01 level of significance. 



Although the attitudes of both instructors and administrators improved, 
only the attitudes of the instructors improved significantly. However, in that 
a score of 48 on the seiticmtic differential represents a neutral attitude. 
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attitudes of both instructors and administrators were above the neutral point 
at the conclusion oil the program. 
Additional Evaluation 

An additional evaluation of the logistics of operating the program in 
Mexico City was done by having the Alabama professors fill out questionnaires 
about their visits (Attachment E) • The results are noted below- 

Travel arrangements . The only problem noted in regard to travel arrange- 
ments was that an airline strike caused one professor to take an alternate 
route. Since this event was beyond the control of the program administrators 
and did not cause any undue hardship, the arrsmgements for travel must be 
considered excellent. 

Housina. Several of the professors remarked that although the housing 
was cwiequate, the msmagement of the apartment building vjas less than adequate. 
One professor went so far as to suggest that the Amberes Apartments never be 
used except as a last resort. 

Local transportation * No problems were noted with local transportation. 
Most professors rode to and from the campus with their students. Not only 
did this provide them with transportation , but it helped them to get better 
acquainted with their students. 

Course arrangements . For the most part, the course arrangements were 
considered exemplary. Particularly,' compliments were paid to the Monterrey 
Tech clerical personnel for their assistance in copying class materials. 

Other comments . Several possible problem areas were noted. Among these 
is that students from the Mexico City campus of Monterrey Tech often were 
expected to carry on with their regular duties while taking classes. This 
did not allow them, in some cases, to have adequate time for preparation. 
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Another possible problem area noted was that of the clinical internships. 
It was felt that the internship needed to be strengthened. Further , several 
professors indicated that some students had complained about the overlap of 
some of the courses. 

Summary and Recommendations 

The evaluation indicates that instructors and administrators participating 
in the master's degree program at Monterrey Tech provided by The University of 
Alabama significantly improved in the areas of achievement, written expression 
in English, and attitude. The courses and the instructors were rated good to 
very good by the students. Further, the arrangements for travel and che 
courses seemed to be more than adequate. 

Even though the program was very successful, several recommendations for 
further improvement are offered. The first is to further improve the 
communication among the professors teaching the courses. In this way, possible 
overlap of courses can be reduced, information about students can be shared/ 
and the professors can be more informed about what to expect when they go to 
Mexico City to teach. 

Based on the evaluation, the only substantive area in urgent need of 
improvement, is the clinical internship. information learned in this project 
needs to be brought to bear on the problem for the future. 

The results of the evaluation support the conclusion that the master's 
degree programs at Monterrey Tech presented by University of Alabama faculty 
was very successful. Much of this success was due to the planning of the 
program, and the cooperation between The University of Alabama and Monterrey 
Tech administrators. 
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MCS STUDENT SURVEY of COURSE/INSTRUCTOR 



OtHlCTlONt pon MAniCINa 
SHUT 

• U« • No. 2 biwii (•«! pwMil. Os 
NOT UM or b«M point p«i. 

•MUitt f«sh mark h««w acid bink, 
fMk tfiouM fUl eir«i« wnv^tmlr. 

•CrMt «l««r4v any a nowa r you wMi 
to ol)«n«»» Makr no itnv maHta. 
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DIRECTIONS 

PlesM rats the instructor of this 
courso according to tha following 
scata: 



1 « Very Poor 

2 « Poor 

3 - Satisfactory 
4 -Good 

5 - Very Good 
Not approprtata 



(VP) 
(P) 
(S) 
(G) 
(VG) 
(NA) 



CLASSROOM NO. 



OOOOOOOOOO 

®®®®®®©®®® 
®®®®®®®®®® 
®®®®®®®®®® 

©®®®®®®®®® 

0000® 00®®® 
0® 0000® 00® 



00 NOT 
MAAK Hfini 



®®®®® 

ooooe 
®®®®® 

0000® 

®®®®® 

0000® 
0000® 

00000 

00000 



VP p s a va MA 

1. AbUity to mak« xh9 count ftitvtnt to tht concorm of cho studtnts O O O O O O 

2. Ability to inswtf nudtntt' qutttiont O O O O O O 

3. Rtipoet for studontt' opinioni O O O O O O 

4. Willingnasi to talk with itudtnts outiidt of eltti O O O O O O 

5» Knovrtadfla of wbjact maitw O O O O O O 

6* Ctartty of objtotivM and foali for tht eotirit ^ O O O O O O 

7. fincottraafmtnt of studant partictpttion O O O O O O 

8. Pratafitstion of courM matarial (mannar of ipo^ch, aUa to haar) .' O O O O O O 

9. Oartty of cfitifla for daianninaiion of gradas O O O O O O 

10. aarity of at<iQn«n*ftts for tha count .O O O O O Q 

Abilhy to ha{p you Itarn btsie atsumptioni in thit fiald O O O O O O 

12. Ability to aiwouragt your sttkina mora information rtlavant to count o o o o o c 

13. AbUity to makt dtu ptfloda vaiuabit O O O O O C 

14. Wlillnfntst to ehtngt « potidon if tvldtnc* wtrranti doing so O O O O O O 

15. Oratfliaation of tht courat O O O O O O 

16. Ratpomt to rtquasts for htip from dia itudanti O O O O O C 

17. Effaotivtnasi in intarprating abitract idaas and thtortti O O O O O O 

la. Abilir/ to dlrtet studtnts to appropriatt rttourcai for inswart O O O O O C 

19. Your rating of tha atsigntd ttxibooki O O O O O O 

20. Your fHling toward tha amount of usigntd work for tha coursa O O O O O O 

21. Your rating of tht tupplamtntary rttdingi O O O O O O 

22. Your rating of tht outsidt of class assignmants and projtcta O O O O O O 

23. Your rating of tha dass dticutsions O O O O C O 

24. Your rtting of tht itcturai O O O O O O 

25. Your rating of tht valut of tht count to you O O O O O O 

20. Compfthtnstvtntu of txaminatJons in measuring whtt you laarntd O O O O C O 

27. Parformanea of initruetor \rt comparison with your othtr initructors O O O O C O 



BACKGROUND INFQRMATIOM 



23. What it your current dats Itvai? 
O Prtfihman O Junior 

O Sophomort O Stntor 

29. What it your itx? O Mala 



O Graduate Student 
O Other 



O Pemaia 



30. What grade do you expect to receive in this coursa? 

OA O C O Pail O Credit O Oo not know 

O a O D O Pats O incomplete 

31. What is your approximate cumuiativa gradt-point ai^eraqa? 

O 3.S0-4.00 O 2.00-2.49 O Lass than 1.00 
O 3.00-3.49 O 1.S0-1.99 O Nona vat-freihman or 
O 2.50*2.99 O i.QO-1.49 cranifsr student 

Cotyri^ht ^ 1973 by National Computer Syittrm. inc. 



ADDITIONAL INSTRUCTOR QUESTIO^fS 

Pill thit in tf the instructor has asked additional 
questions. 

A3C0BPGHI J 

1 ococoooooc 

2. ....cccoocoooo 

3 OOOOOOOOOO 

A OOOOOOOOOO 

s OOOOOOOOOO 

6 OOOOOOOOOO 

7 OOOOOOOOOO 

8 OOOOOOOOOO 

9 OOOOOOOOOO 

10 OOOOOOOOOO 

NCS TfcnsOotiC T1344-343 
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ATTACHMENT B 



Supplementary Questions for 
NCS Student Survey of Course/Instructor 



1, How would you characterize the effort you put forth in this course? 

A. Very little 
• B. Less than average 

C. Average 

D. More than average 

E. Very much 

2. The course schedule was: 

A. Inconvenient 

B. Moderately convenient 
Convenient 

3* The classroom and equipment were: . 



A. Inadequate 

B. Moderately adequate 

C. Adequate 
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ATTACHMENT C 

INSTRUCTIONS FOR 
COLLECTING POSTTEST WRITING SAMPLES 

The purpose of collecting the writing samples is to assess the 
growth of the people in the Monterrey Tech Program in their facility 
with the English language and their understanding of the ITESM missions 
and purposes. Pretest samples were collected when they began their 
programs last summer. 

Although th^ exercise is not timed, it should not take over an hour. 

Be sure everyone has plenty of paper before you begin. When everyone is 

ready, please read (or paraphrase if you wish) the following directions : 

If you recall, last summer each of you wrote a paper on the missions 
and purposes of the ITESM. In order to assess your growth in the use of 
the English langxiage and your further understanding of the missions and 
rolea of the ITESM, we would like you to repeat the exercise. Please 
^Ite a paper in English addressing the following two questions: 

1. What do you believe are the missions and purposes of the 
ITESM in Mexico? 

2. What do you believe are your roles and functions as they 
relate to the ITESM missions and functions? 

You may take all of the time you wish and write as many pages as 
you think it takes to answer adequately the questions. When you finish, 
turn in your paper to me and take a break. I will call back when we are 
ready to continue. 

Please write the two questions on the board so they can refer to 
them as they wish. You may wish to do this near the last hour of a 
period so they can go home afterwards. Do not let them take this 
asalgiu&ent home* It must be done In class. 
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Attachment D 



NAME 



Below are pairs of descriptive adjectives. The purpose of this 
Instrument Is to Judge your feelings about your job based on the following 
sets of adjectives. The results will be reported for all responses 
coniblned. Your responses will be kept anonymous. Please mark each 
adjective pair using the following scale: 

1. Your present position Is very much related to the first 
adj active. 

4. Your present position Is midway between the two adjecnlves. 

7. Your present position Is very much related to the last 
adjective. 



My job 



Good 


1 2 


3 


4 


5 


6 


7 


Bad 


Incomplete 


1 2 


3 


4 


5 


6 


7 


Complete 


Pleasurable 


1 2 


3 


4 


5 


6 


7 


Painful 


Unsuccessful 


1 2 


3 


4 


5 


6 


7 


Successful 


Meaningful 


1 2 


3 


4 


5 


6 


7 


Meaningless 


Unimportant 


1 2 


3 


4 


5 


6 


7 


Important 


Congenial 


1 2 


3 


4 


5 


6 


7 


Quarrelsome 


Ignorant 


1 2 


3 


4 


5 


6 


7 


Educated 


Useless 


1 2 


3 


4 


5 


6 


7 


Useful 


Perfect 


1 2 


3 


4 


5 


6 


7 


Imperfect 


Unullllng 


1 2 


3 


4 


5 


6 


7 


Willing 


Comfortable 


1 2 


3 


4 


5 


6 


7 


Uncomfortable 
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ATTACHMENT E 



Course Instructor's Questionnaire 



Name 



Course (s) 



Date 



Please give your opinion about the following items relevant to your 
teaching assignment in Mexico. 

!♦ Travel arrangements: 



2. Houslx\g: 



3. Local transportation: 



4, Course arrangements (please include teaching facilities and equipment) : 



5. Any other comments: 
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